Metropolitan Sewerage District 

OF BUNCOMBE COUNTY, NORTH CAROLINA 



June 6,2013 

Mr. Chuck Cranford, Regional Supervisor 
Surface Water Protection Section 
Division of Water Quality 
Asheville Regional Office 
2090 U.S. Highway 70 
Swannanoa, NC 28778 

Subject: Notice of Violation 
NOV-2013-DV-0120 
Permit No. WQCS00004 
Buncombe County 

Dear Mr. Cranford: 

This letter is in response to the subject NOV issued in your letter of May 28, 2013. 
Below is a synopsis of the cause of the SSO, the response, and further action that was 
taken. 

MSD contracted with Gilbert Engineering Company to replace one of the three 
pumps at the influent pump station at the Treatment Plant. This pump is 
approximately 25 years old and is reaching the end of its useful life. Unfortunately 
the fail-safe procedures agreed to in the preconstruction conference were not 
followed, in addition to the fact that MSD Operations Staff were not notified of the 
work being started. 

On April 30 th , the Contractor (construction field staff) proceeded to replace the 
pump valve packing without following the fail-safe procedure established 
beforehand by MSD. The procedure involved shutting down the pumps and 
draining the system in order to work on the valve. An investigation was conducted 
(report attached) to determine why the procedure was not followed, and why MSD 
Operations staff were not notified. 

Once the incident occurred, MSD Plant Operations and Collection System Operations 
Staff were notified and a "code-red" was issued. MSD O&M personnel immediately 
responded to clear and pump out the area, and then to quickly repair the damage 
that had been done. 

-Protecting Our Natural Resources- 




RECEIVED 

JUN - 6 2013 



DWQ/Surface Water Protection Section 

Asheville Regional Office 



2028 RIVERSIDE DRIVE, ASHEVILLE, NORTH CAROLINA 28804 TELEPHONE: (828)254-9646 FAX: (828)254-3299 WEBSITE: www.msdbb.org 
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This was a valiant effort in tough conditions. The MSD Operations Staff showed 
great resolve and focus to get the situation under control, repair the valve, and 
minimize the down time. 

The Spill lasted from 8:35 am until 2:45 pm. The spill rate was 0.95 million gallons 
per hour (based upon system flow monitors) and the river flow rate was 192.5 
million gallons per hour. The total spill volume was 5.86 million gallons. 

Once the repair was made and the overflow ceased; System Services personnel 
rallied to clean up all spill areas (basically adjacent to manholes between the 
administration building and Woodfin Park) by removing contaminated surface soils 
and then applying lime to disinfect and then putting down straw and mulch. This 
was completed for all areas by 8pm that same day (Tuesday). 

MSD immediately began water quality testing at several spots along the river in 
concert with NCDENR Staff, who performed tests as well. This testing lasted from 
when the leak started through Friday afternoon. Spill updates and sampling 
information were shared with the environmental and health officials, the MSD 
Board, the press, and several stakeholder groups along the French Broad River. 
Testing results indicated that the river quality from the Plant to Hot Springs had 
returned to normal by Thursday morning. 

After the spill itself Was addressed, MSD followed-up to guard against any further 
occurrence on several fronts. First MSD conducted an investigation of the 
circumstances surrounding the accident. As a result MSD has taken a personnel 
action against the employee that was involved with the contractor. MSD put the 
Contractor on notice of their responsibility as well. Critical staff met to go over 
lessons learned from the entire incident as well as sharing the investigation report 
with all staff, since human error was involved. 

There are 3 high service pumps (35 MGD each), one lead, one lag and one as back-up 
standby. In addition, MSD has a 2MW standby generator (currently being upgraded 
to 4 MW) and 2 different feeds from the Progress energy sub station. The current 
station utilizes a dry pit design for the pump and motor units with a 210 GPM sump 
pump for leaks within the dry pit area. In addition MSD has both a hydraulic driven 
and diesel driven emergency portable pumps which were used to dewater the area. 

The cause of the accident was not following the shut -off procedures prior to 
working on the valve. However, in an effort to provide backup to the backup, and in 
an effort to expand the safety margin in time before a potential spill might occur, 
MSD has engineered and purchased a much larger sump pump of 700 GPM that will 
remain in the dry pit permanently. This measure will more than triple the 
dewatering capacity of the existing sump pump. In addition, MSD has purchased 2 
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additional portable 12 -inch diesel-driven emergency pumps and has them staged 
at the front of the plant which can also can be used for a full pump around of the 
Carrier Bridge pump station. 

Please find the attached information: 

1. Four news releases (two on April 30 th , one May 1st and a publication May 6 th ) 

2. Wednesday May 1 st Spill Recap/Update - which was emailed directly to 
Health and Environmental Officials, the MSD Board, French Broad Users, and 
the press. 

3. MSD Spill River Sampling, French Broad River, Results as of Thursday 
May2nd 

4. River Sample Data Spread sheet April 30 th through May 3, 2013. 

5. MSD Sewage Spill - 4/40/13 Investigation and Background, 

6. Photos 

This was a very unfortunate accident. MSD takes seriously its role and record of 
environmental stewardship. 

Please call with any questions or if you require additional information. 
Sincerelv 

r 




Thomas E. Hartye, Y.E, 
General Manager 




TEH:sh 
Attachments 



ASHEVILLE 

CITIZEN-TIMES 



VOICE OF THE MOUNTAINS • OTIZEN-TIMES.com 



AFFIDAVIT OF PUBLICATION 



NOTICE OF DISCHARGE OF 
UNTREATED SEWAGE 

The Metropolitan Sewerage District of 
Buncombe County notifies the public of an 
untreated sewage, discharge at 2028 Riverside 
Drive in Asheville. An estimated 5,860,000 
gallons of untreated sewage began at 8:35 a.m., 
April 30, 2013. The duration of the 'discharge 
was 6 hours. The cause of the discharge into 
the French Broad River was due to a valve 
blow out at the MSD main pump station that 
feeds the plant in Woodfin while a contractor 
was performing maintenance on it: The cause 
of the discharge has been corrected and the 
affected area has been checked for any residual 
effects. This notice was required by North 
Carolina General Statutes Article 21 Chapter 
143.215.C. 

If you would like more information, please 
call Ken Stines at MSD - 255-0061 

May 5, 2013 (1156) 



AK-0000332S40 



BUNCOMBE COUNTY 

ss. 

NORTH CAROLINA 

Before the undersigned, a Notary Public of said County and 
State, duly commissioned, qualified and authorized by law 
to administer oaths, personally appeared Velene Fagan, 
who, being first duly sworn, deposes and says: that she is 
the Legal Billing Clerk of The Asheville Citizen-Times, 
engaged in publication of a newspaper known as The 
Asheville Citizen-Times, published, issued, and entered as 
first class mail in the City of Asheville, in said County and 
State; that she is authorized to make this affidavit and 
sworn statement; that the notice or other legal 
advertisement, a true copy of which is attached hereto, was 
published in The Asheville Citizen-Times on the 
following date: May 5 th , 2013. And that the said newspaper 
in which said notice, paper, document or legal 
advertisement was published was, at the time of each and 
every publication, a newspaper meeting all of the 
requirements and qualifications of Section 1-597 of the 
General Statues of North Carolina and was a qualified 
newspaper within the meaning of Section 1-597 of the 
General Statutes of North Carolina. 



Signed this 6 th , day of May, 2013 



RECEIVED 



JUN - 6 2013 



DWQ/Surface Water Protection Section 
AshevilleRejional Office 




(Signature of person making affidavit) 



Sworn to and subscribed before me the 6 th day of May, 
2013. 




My Commission expires the 5 th day of October, 2013 ^ JP Y E 



(828)232-5830 | (828) 253-5092 FAX 
14 0. HENRY AVE. | P.O. BOX 2090 | ASHEVILLE, NC 28802 | (800)800-4204 



TARY 



GAhNEIT 



April 30, 2013 Spill Recap/Update 
(Wednesday May 1 st ) 



The spill lasted from 8:35 am till 2:45 pm. The spill flow rate was 0.95 
million gallons per hour. 

The river was flowing at 192.5 million gallons per hour. 

The System Services Folks rallied to have all spill areas cleaned up by 8 
pm last night. Basically removing surface soils, adding lime for 
disinfection and adding back straw and mulch for restoration. 

River/Spill Plume velocity: 2.9 miles per hour 

Marshall (16.7 river miles) 5.8 hours - passed by at approximately 9 pm 
last night (Tues. 4/30). 

Hot Springs (32 miles) 11 hours - passed by at approximately 2 am this 
morning (Wed. 5/1). 

We are testing several spots along the river in concert with NCDENR. 
Unfortunately these coliform bacteria tests take 24 hours to perform. We 
will use these tests to give the all clear. 

I just recently gave interviews relating this information and answered 
questions for the TV press WLOS and News 4. 



Please forward this on to whomever you see fit and call me with any questions or input. 



MSD Spill River Sampling 
French Broad River 
Results as of Thursday May 2nd 



Typical untreated sewage Fecal coliform count is between 100,000 to 
1,000,000 (cfu/100ml). 



Results: 

Tuesday April 30 th . while the spill was passing by, at 4:35 pm the count was 
6700 at Ledges Park. 



Wednesday May 1 st at 10:13 am the count was down to 430 at Ledges Park 
which is near normal range. 

Wednesday May 1 st at 11:32 am the count was down to 460 at Walnut Island 
Park which is near normal range. 

Wednesday May 1 st at 12:10 pm the count was down to 320 at Hot Springs 
Bridge which is near normal range. 



Thursday May 2 nd at Marshall Bridge the count was at 148 in normal range. 
Thursday May 2 nd at Hot Springs the count was at 87 in normal range. 

Upstream sample at Craggy Bridge which was unaffected by spill , the counts 
ranged from 310 to 520 on Wednesday May 1 st . 
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NOTICE OF DISCHARGE OF 
UNTREATED SEWAGE 

The Metropolitan Sewerage District of 
Buncombe County notifies the public of an 
untreated sewage discharge at the main treatment 
plant in Woodfin due to an equipment failure 
beginning at 8 a.m. this morning. The District is 
currently working to get the main pumps back 
online. Please refrain from using the French Broad 
River downstream at the plant for recreation at this 
time. Treatment should be restored in the next few 
hours. 

If you would like more information, please 
call Ken Stines at MSD - 255-0061 



JUN - 6 2013 



DWQ/Surface Water Protection Section 
Asheville Regional Office 
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NOTICE OF DISCHARGE OF 
UNTREATED SEWAGE 



The Metropolitan Sewerage District of 
Buncombe County notifies the public of an 
untreated sewage discharge at the main treatment 
plant in Woodfin due to an equipment failure 
beginning at 8 a.m. this morning. The District is 
currently working to get the main pumps back 
online. Please refrain from using the French Broad 
River downstream at the plant for recreation at this 
time. Treatment should be. restored in the next few 
hour's. 



F 3 
B. 




FOLLOW-UP 
NOTICE OF DISCHARGE OF 
UNTREATED SEWAGE 

The Metropolitan Sewerage District of 
Buncombe County notifies the public of an 
untreated sewage discharge at the main treatment 
plant in Woodfin due to an equipment failure 
beginning at 8 a.m. this morning. The District 
restored service to the station at 2:40 p.m. Please 
refrain from using the French Broad River 
downstream at the plant for recreation at this time. 

If you would like more information, please 
call Ken Stines at MSD - 255-0061 



RECEIVED 



JUN - 6 2013 



DWQ/Surface Water Protection Section 
Asheville Regional Office 
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FOLLOW-UP 
NOTICE OF DISCHARGE OF 
UNTREATED SEWAGE 

The Metropolitan Sewerage District of 
Buncombe County notifies the public of an 
untreated sewage discharge at the main treatment 
plant in Woodfin due to an equipment failure 
beginning at 8 a.m. this morning. The District 
restored service to the station at 2:40 p.m. Please 
refrain from using the French Broad River 
downstream at the plant for recreation at this time. 




Metropolitan 
Sewerage 
District 



Contact: Ken Stines 
Metropolitan Sewerage District 
Phone: (828) 255-0061 
Fax: (828) 251-9040 



Post Office Box 8969 
2028 Riverside Drive 
Asheville, North Carolina 28814 



News Release 



NOTICE OF DISCHARGE OF UNTREATED SEWAGE 



The Metropolitan Sewerage District of Buncombe County notifies the public of an 
untreated sewage discharge within the collection system of our district. The 
cause of the discharge has been corrected and the affected area has been 
checked for any residual affects. This notice was required by North Carolina 
General Statutes Article 21 Chapter 1 43.21 5.C 



Location of discharge: 2028 RIVERSIDE PR - ASHEVILLE 
Estimated volume entering surface waters: §880000 



Time and date of discharge: 4/30/2013 8:35:00 AM 



gallons 



Duration of discharge: 
Surface water entered: 



6 hrs. 

FRENCH BROAD RIVER 



RECEIVED 

JUN - 6 2013 



DWQ/Surface Water Protection Section 
Aslieville Regional Office 



Cause of discharge: 



A VALVE BLEW OUT AT OUR MAIN PUMP STATION 
THAT FEEDS THE PLANT HERE IN WOODFIN WHILE 
A CONTRACTOR WAS PERFORMING MAINTENANCE 
ON IT. 



Comments: 



THE RIVER WAS FLOWING AT 192.§ MILLION 
GALLONS PER HOUR: 



Notification required within 48 hours of Determination 



05/01/13 11:49 FAX 8282519040 
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Contact: Ken Stines 
Metropolitan Sewerage District 
Phone: (828)255-0061 
Fax: (828) 251-9040 



Post Office Box 8969 
2028 Riverside Drive 
Asheville, North Carolina 28814 



News Release 



NOTICE OF DISCHARGE OF UNTREATED SEWAGE 



The Metropolitan Sewerage District of Buncombe County notifies the public of an 
untreated sewage discharge within the collection system of our district. The 
cause of the discharge has been corrected and the affected area has been 
checked for any residual affects. This notice was required by North Carolina 
General Statutes Article 21 Chapter 143.215.C 



Location of discharge: 2028 RIVERSIDE PR - ASHEVILLE 
Estimated volume entering surface waters: 5860000 

gallons 

Time and date of discharge: 4/30/2013 8:35:00 AM 



MSP Sewage Spill - 4/30/13 / Investigation and Background 



On April 30, 2013, Gilbert Engineering Company ("Gilbert"), an independent contractor 
hired by MSD, began work on a project to remove and reinstall pump number 3 at the influent 
pump station. The specific task being performed by Gilbert on April 30, 2013, was to remove 
and replace packing in the existing isolation knife gate. This was the first step in the removal and 
replacement of the pump. 

MSD staff had removed and replaced this same type of packing, approximately 6 to 7 
years earlier, on another identical pump, as a maintenance item. Bob Triplett, MSD employee, 
was involved in that project. 

Because Gilbert was going to do the pump replacement, it was determined that they 
should do the removal and replacement of the packing around the knife gate valve. Gilbert 
submitted a proposal to MSD dated May 14, 2012 to replace one influent pump. Gilbert 
submitted a modified proposal dated April 1, 2013. Gilbert, and in particular, its supervisor, 
Chuck Miller (who was formerly employed by Hickory Construction), had done work for MSD 
in the past and had worked with Mike Schraven, the MSD construction inspector assigned to 
coordinate this job with Gilbert. 

MSD staff discussed this project in March and in April 2013. Staff agreed that the plant 
should be shut down when the work was being done. Peter Weed, Dennis Lance, Jay Cormier, 
Mike Schraven (MSD) and Hunter Carson, MSD engineer, met with Chuck Miller, the 
construction superintendent on this job for Gilbert on March 19, 2013 at MSD. At the meeting, a 
procedure for repacking the knife gate valve was discussed. The procedure involved turning off 
all the influent pumps (which pump untreated wastewater up into the plant) beginning at 8 A.M. 
The time between 8 and 10 AM was chosen because MSD studied flow patterns, and determined 
the lowest flows occurred during that time period. 

When the pumps were turned off, Gilbert would remove the existing packing and put new 
packing in place. While this work was being done, the plan was to use the French Broad 
Interceptor sewer line (main line leading to the plant) for storage in order to facilitate the work. 

Collection System Staff were directed to do a "dry run," to see how long it would take 
wastewater to back up into the upstream manholes. The dry run was done on April 5, 2013 under 
the supervision of Hunter Carson, an MSD engineer and Ken Stines, Co-Director of System 
Services. Peter Weed, plant manager, reported in the Division Director's meeting on April 10, 
2013, that Ken Stine's crew had determined that an SSO would occur at the first upstream 
manhole 48 minutes after shutdown. The procedure as to how the work was to be done and the 
window of time to do the work were communicated to Mike Schraven and to Chuck Miller of 
Gilbert. 



On Tuesday, April 30, 2013, at approximately 7:15 A.M., Mike Schraven and a crew 
from Gilbert, consisting of a foreman, Don Sigman, and his two sons, Keith and Steve, went to 
the influent pump station to remove and replace the existing packing around the knife gate valve 
on influent pump # 3. The yoke of the valve had been removed the previous Wednesday, April 
24, as had the stem and thimble. Mike Schraven did not tell any of the plant personnel or his 
supervisor, Ed Bradford about the work being done on the pump that day. 

Chuck Miller, the supervisor for Gilbert, was aware the work was going on the morning 
of April 30, but he was not on site when the work began. He did not talk with anyone that 
morning. He was not worried about the work. He came to the site shortly after getting a call from 
Don Sigman that an accident had occurred. When he arrived he heard that a pump had been left 
running and asked why. 

Mike Schraven said he thought it was okay to do the work without shutting down the 
other influent pumps (1 and 2) and thought that the knife plate was stable, because he believed 
there was no pressure on the backside of the knife gate valve. He thought this was the case 
because the water valve on the pump volute was open and no water was coming out of it. The 
decision to proceed with the work with pump running was made by Mike Schraven and Don 
Sigman. Shortly after the work began, the knife plate shot out of the knife valve housing, hit the 
wall and then fell on one of the Sigman sons, who had been knocked to the floor in the process. 
He hit his head on the wall as he fell. His brother, Keith, and Bob Triplett removed the gate plate 
from his chest, and then walked him upstairs. Mike Schraven stated that he heard a crash after 
the work began. He immediately shut off the pumps, noticed that a worker was being helped and 
called Dennis Lance at the plant. [This version of the facts was provided by Mike Schraven 
immediately after the accident and during an interview with the Director of CIP, the Plant 
Director and the General Manager]. 

During an interview with the Director of Personnel, the CIP Director and the General 
Manager on Friday, May 3, 2013, and in a memorandum e-mailed to the General Manager on 
May 5, 2013, Mr. Schraven stated that it was never the intention to do the work with any of the 
pumps running, that he was expecting the Gilbert foreman to tell him when they needed to turn 
the pumps off so he could call the plant and tell them to turn the pumps off , that he had his back 
turned to the workers looking at the plumbing for a seal water pump system that MSD staff were 
about to rework when he heard clanging steel, water rushing and screaming. He turned around to 
see water rushing from the slide gate throat and workers struggling to get out from under the 
scaffolding. He ran upstairs, killed the pumps, and called Dennis Lance at the treatment plant. 
Mr. Schraven did not notify plant staff that the work on the pumps was being done on April 30, 
2013. He said that Gilbert said they were ready to do the work, so he, Mike, said okay. He thinks 
he might have mentioned it on Monday, April 29, 2013, during discussions concerning a separate 
project at the treatment plant. Neither Dennis Lance, nor Ed Bradford, nor Peter Weed recall 
Mike Schraven mentioning this to them or discussing it on Monday, the 29 th ' 

John Gilbert, the owner of the Company, mentioned that he discussed the procedure for 
this work with his staff, including Don Sigman, at their usual Monday morning meeting to 
discuss work. 



Bob Triplett said he told Donald Sigman the week before, how MSD had done the work 
with the pumps off the last time MSD replaced the packing around the knife gate valve. Mr. 
Triplett was working on another project on the lowest floor of the pump building, adjacent to 
where the work was being performed. At about 7:45 A.M., he noticed that the Gilbert crews were 
about to perform the repacking. He asked Don Sigman when they were going to turn the other 
Raw Pump off. Don's reply was that he reckoned they were not. Bob Triplett warned them that 
they better be prepared to get very wet if the silt above the knife gate did not hold. He then stood 
by just in case they would need his help for any reason. Minutes later, the knife gate started 
gushing water, the plate itself flew out hitting the one worker, the wall behind and then dropping 
down on top of the same worker. Bob then assisted in removing the heavy steel plate and 
supporting the Gilbert employee upstairs. 

Once the incident occurred, MSD Plant Operations and Collection System operations 
were notified and a "code-red" was issued. MSD O&M personnel immediately responded to 
clear and pump out the area and to repair the damage that had been done. 
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